Distributed Variable Sample-Size Stochastic Optimization With Fixed Step-Sizes

In this article, we consider distributed stochastic optimization over randomly switching networks, where agents collaboratively minimize the average of all agents’ local expectation-valued convex cost functions. Due to stochasticity in gradient observations, distributedness functions, and randomness communication topologies, algorithms with an exact convergence guarantee under fixed step-sizes have not been achieved yet. This work incorporates variance reduction scheme into tracking algorithm, gradients are estimated by averaging across a variable number sampled gradients. With identically independently random network, show that iterates converge almost surely same optimal solution step-sizes. When global function is strongly sample size increases at geometric rate, prove geometrically unique solution, establish iteration, oracle, complexity. The algorithm performance, including rate complexity analysis, further investigated constant polynomially increasing size. Finally, empirical performance illustrated numerical examples.